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ABSTRACT 

The Illinois Course Evaluation Questionnaire (CEQ), is 
an instrument used to collect student attitudes towards a course. .Its 
purpose is to enable faculty members to collect evaluative 
information about their traching. The data are collected and 
processed by section but may also be processed by course, department, 
college, etc. The student responses are anonymous, and two copies of 
the questionnaire results are returned to the instructor only. He may 
submit one copy, if he chooses to do so, to his department chairman 
for consideration in the evaluation of his teaching. . Each 
instructor's results are compared to an appropriate norm group. 
Comparisons are made with other instructors of his own academic rank, 
with thoses at the same course level (i.e., 100 level =freshinen, 200 
level= sophomore, 300 level=junior and senior, 400 level=graduate) , . 
with other instructors in his particular department or college, as 
well as with all courses at that university to all courses that have 
used the CEQ throughout the United States. Conferences relative to 
the interpretation of results may be arranged at the instructor" s 
convenience. . (Author) 
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ABSTRACT 



The Illinois Course Evaluation Questionnaire (CEQ) is an in- 
strument used to collect student attitudes towards a course. Its 
purpose is to enable faculty members to collect evaluative infor- 
mation about their teaching. The data are collected and process- 
ed by section but may also be processed by course, department, 
college, etc. The student responses are anonymous, and two copies 
of the questionnaire results are returned to the instructor only. 
He may submit one copy, if he chooses to do so, to his department 
chairman for consideration in the evaluation of his teaching. 

Each instructor's results are compared to an appropriate norm 
group. Comparisons are made with other instructors of his own 
academic rank, with those at the same course level (i.e. , 100 
level = freshmen, 200 level = sophomore, 300 level = junior and 
senior, AOO level = graduate), with other instructors in his par- 
ticular department or college, as well as with all courses at that 
university to all courses that have used the CEQ throughout the 
United States. Conferences relative to the interpretation of re- 
sults may be arranged at the instructor's convenience. The eval- 
uation service is free of charge to instructors at the University 
of Illinois and is available upon request from the Measurement 
and Research Division, Office of Instructional Resources, 307 
Engineering Hall, Urbana, Illinois 61801, (2.7) 333-3490. 
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ILLINOIS COURSE EVALUATION QUESTIONNAIRE (CEQ) 
RESULTS INTERPRETATION MANUAL 
FORM 66 AND FORM 32^ 

If one assumes that the purpose of education is to change 
student behavior as a result of some definite course of instruc- 
tion, then an objective of educational research should be to 
determine what procedures or techniques best produce the desired 
behavioral changes. If the desired behavioral changes can be 
identified and defined, then the educational researcher can develop 
instruments to measure them. 

Let us also assume that if one does in fact change student 
behavior in the specified direction, as a result of a course of 
instruction, then that course has been effective. If that course 
has been effective, then there could be a large number of elements 
in that course contributing to its effectiveness, such as the 
instructor, textbook, homework, course content, method of instruc- 
tion, student interest, student attention, general student 
attitude towards the course, etc. 

Assuming that all of the elements enumerated above can affect, 
directly or indirectly, student behavior in a course, and assuming 
that the students are the only ones who are constantly exposed to 
those elements, then they appear to be the most logical evaluators 
of the quality and effectiveness of the course elements. In addi- 
tion, student opinions should indicate areas of rapport, degrees 
of communication, or the existence of problems and thereby help 
instructors as well as educational researchers describe and define 
the learning environment more concretely and objectively than they 
could through other types of measurements. 

There are various ways of sampling student opinion. Some 
useful information can be derived simply by detenaining the number 
of students who agree or disagree with certain statements about the 
course. Or, sometimes, it proves useful to ask students to write 
short essays about the course in order to obtain information about 
their experiences under specific instructional situations • Such 



^Form developed and copyrighted by the late Richard E. Spencer, 
1965. 



individualized procedures do not, however, provide an opportunity 
to compare the results of one course with results of another. 
Measurement is more useful when comparative results are available. 
More adequate interpretation may occur when: (a) the data have 
been collected in a standardized fashion with appropriate atten- 
tion given to sampling, reliability, and validity, and (b) many 
instructors and instructional programs have been measured with the 
same instrument so that comparisons can be made. 

Therefore, the primary purpose in developing the Illinois 
Course Evaluation Questionnaire (CEQ) was to devise an instrument 
which could elicit student opinions about a standardized set of 
statements relative to certain -dardized aspects of an instruc- 
tional program, and to develop norms which would enable an 
instructor to adequately compare his results with the results of 
other instructors. 

This manual has been prepared to help instructors who use the 
Illinois Course Evaluation Questionnaire (CEQ) to interpret their 
results. A facsimile of the computer generated CEQ report of 
results is included which can be referred to in the following 
discussion. There may be a minimum of two pages or a maximum of 
four pages of results, depending upon the number of optional 
questionnaire items used, i.e., on Form 66 (items 51-100) or on 
Form 32 (items 25-76). An example of Form 66 and Form 32 appear 
in Appendix A and B, respectively. 

For both forms instructor and course identification informa- 
tion is presented on the top of the first page of the results, 
followed by the statistics for the first portion of the standard 
CEQ items. For examples of the first page of Form 66 see Figure 1 
and for Form 32 see Figure 2. The second page of the results 
presents the statistics for items 29-50 for Form 66 and item 21-24 
for Form 32 and summary results. If optional items are used, they 
would appear on additional pages as shown in Figure 3. 



Identification Information 

(1) COURSE AND INSTRUCTOR DATA 

Inatmctor Name: The instructor's >st name appears on 
the first line of printing. It is circled on the sample 
output. 

Course Code: A five-digit numeric course identification 
code is presented on the first line, just above "Expected 
Grade." 

Course Name and Level: This is presented on the right 
side of the first line. 
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Additional Identifying Infomation: This may follow 
the course ncrnie and level if it is submitted by the 
instructor (i.e., section code, etc.). 

(2) STUDEl'JT DATA 

Expected Grade in the Course: This is presented as 
the percentage of students who expect an A, B, C, D, 
or E, and who omitted (OMIT) the question. 

Sex of Student: The percentage of males and females. 

Student Status: The percentage of freshmen, sophomores, 
juniors, seniors, graduate students, and others. 

Required or Elective: The percentage of students taking 
the course to fulfill a requirement or who <.hose it as 
an elective. 

College Affiliation: The percentage of students by 
their college affiliation. 

Sample Size: The number of students responding to the 
CEQ is presented below the summary results on the 
second page. 

The student data are presented so that the instructor 
may compare the student characteristics of one class 
section with another. These characteristics may lead 
to different interpretations of results since some 
class sections are composed of students who are quite 
different from other sections. If the CEQ results 
differ from one section to another, the student 
characteristics may lead to a more adequate under- 
standing or interpretation of these differences. 



Item Responses 1-50 or 1-24 

Several columns of figures reporting the students* responses 
for each item are presented following the student data. Due to 
the limited printing space, one must consult the CEQ Forms 
(Aopendix A or B) to determine what the specific items are. 

For explanatory purposes, the sample results presented in 
Figures 1, 2, and 3 have lettered codes printed at the bottom of 
each of the columns. The interpretation of the lettered columm; 
is as follows: 

A: Contains the ITEM number, so one can refer back to the 
CEQ Form for the specific item statement. 

B; The proportion of students strongly agreeing (SA) with 
this item. 
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C: The proportion of students agreeing (A) with this 
item. 

D: The proportion of students disagreeing (D) with 
this item, 

E: The proportion of students strongly disagreeing 
(SD) with this item. 

F: The proportion of students leaving this item 
blank (OMIT). 

C: This column indicates the most favorable response 
for each item, either SA or SD. On Form 66, 24 of 
the items are assigned the most fa>rorable response 
of SA and 26 are assigned the most favorable 
response of SD, On Form 32 all of the items are 
assigned the most favorable response of SA. 

H: The numerical average (MEAN) of the SA, A, D, and 
SD responses is presented for each item. Tne 
MEAN is obtained by weighting the positively 
stated items, SA = 4, A = 3, D = 2, and SD = 1, 
and the negatively stated items, SA = 1, A = 2, 
D = 3, and SD = 4. 

I: This column contains the standard deviation (S.D.) 
of the responses for each item. 

J: This column presents the all university norm decile 
(DECL) , i.e., a comparison of this class' HEM^ 
responses with those obtained in other classes 
throughout the university. This comparison group 
is referred to as the '*norm group.'' Deciles range 
from a low of 0 to a high of 9 and may be inter- 
preted as follows: 

0 indicates that the course MEAN falls in 
the lowest 10 percent of the norm group, 

1 indicates that there are 10 to 19 percent 
of the norm group who received lower means, 

2 indicates that there are 20 to 29 percent 
of the norm group who received lower 
means , 

and so on, through 9. 

The distribution of class results in the norms can 
be exemplified by using a distribution based on an 
item with a MEAN of 2.8 and a Standard Deviation 
of .40 as follows: 
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The distribution of scores was divided into ten 
equal size groups (hence the term decile). The 
highest 10% of the classes were assigned the 
decile of 9; the next 10% the decile of 8; etc. 
Each decile contains an equal number of classes. 
The same process was followed for the subscores. 
Interpretation of these deciles will be discuss (d 
later in the manual, however, the actual norm 
decile intervals for the items are presented in 
Appendix C. 

K: The Decile Profile is presented in this column by 

printing an asterisk corresponding to the appropri ite 
decile. The asterisks which are found on the 
right side of the column (deciles of 7, 8, or 9) 
should be considered "good.'* Those to the left 
(deciles of 0, 1, or 2) should be considered "pjor." 
Those in the middle (deciles 3, 4, 5, or 6) 
should be considered *' average 



Subscale Responses 

The second page of the results presents the questionnaire 
SUBSCORES. You will note that there are six subscores and one 
total score. The items which are grouped into each subscale are 
presented in Table 1. The total score is a score made up of all 
items. 

L: The number of ITEMS comprising the subscore. 

M: The percentage of students who responded (RESP) to 
the items included in the subscale. 

N: The MEAN response based on the items in each subscale. 

0: The standard deviation (S.D.) of the responses. 

P: The reliab^.lity (REL) is based on a split-half 

correlation (odd-evpn) and indicates the confidence 
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TABLE 1 



CEQ Items Grouped by Subscales 

01. General Course Attitude 

2. It was a waste of time.* 

3. Overall, the course was good. 

11. Not much was gained by taking this course.* 

20. The course increased my general knowledge. 

25. It was a very worthwhile couroe. 
29 One .'f my poorest courses.* 

3^. "our"? %'as not very helpful.* 

49. The .';ourse was quite useful. 

02. Method of Instruction 

1. I \v*arn more when other teaching methods aie 
used - * 

6. More courses should be taught this way. 
8. I would have preferred another method ol 
teaching in this course.* 

27. The way in which this course was taught 
results in better student learning. 

36. I think that the course was taught quite wel 

37. I would prefer a different method of ins true 
tion.* 

48. Another method of instruction should have 
been employed.* 

50. I would take another course that was taught 
this way . 

03. Course Content 

13. The course material seemed worthwhile. 
19. The content of the course was good. 

26. Some things were not explained very well.* 

28. The course material was too difficult.* 
30. Material in the course was easy to follow. 

39. At times I was confused.* 

40. Excellent course content. 

44. The content of the course was too elementary 



*These items appear on Form 66 only. 



TABLE 1 (Continued) 



04. Interest and Attention 

7. The course held my interest. 

9. It was easy to remain attentive. 

14. It was difficult to remain attentive.* 

22. Held my attention throughout the course. 
24. Uninteresting course.* 

35. It was quite interesting. 

45. Some days I was not very interested in this 
course. * 

46. It was quite boring.* 

05. Instructor 

5. The instructor seemed to be interested in 
students as persons. 

10. The instructor did not synthesize, integrate, 
or summarize effectively.* 

12. The instructor encouraged the development of 

new viewpoints and appreciations. 
15^ Instructor did not review promptly and in such 

a way that students could understand their 

weaknesses . * 

18. The instructor had a thorough knowledge of his 
subject matter. 

23. The demands of the students were not considered 
by the instructor.* 

31. The instructor seemed to consider teaching as 
a chore or routine activity.* 

47. The instructor exhibited professional dignity 
and bearing in the classroom. 

06. Specific Items 

4. The textbook was very good. 

16. Homework assignments were helpful in under- 
standing the course. 

17. There was not enough student participation 
for this type of course.* 

21. The types of test questions used were good. 

32. More outside reading is necessary.* 

33. Course material was poorly organized.* 
38. The pace of the course was too slow.* 

41. The examinations were too difficult.* 

42. Generally, the course was well organized. 

43. Ideas and concepts were developed too rapidly.* 



one can place in the results for this group of 
students. Reliability figures over .90 can be 
considered very good; .70 and above are probably 
acceptable; and below .70 are questionable. Re- 
liability figures below .70 may be due to small 
samples of students. 

Q: The RANK norm compares the course to all courses 
of instructors at cliat same rank. The actual 
norm decile intervals are presented in Appendix D. 

R: The LEVEL norm compares the course to all courses 
at that same course level (i.e., Freshmen, Sopho- 
more, Junior, Senior, or Graduate). The actual 
norm decile intervals are presented in Appendix E. 

S: The institution (INSTI) norm compares the course 

to all courses at that university (i.e.. University 
of Illinois). The actual norm decile intervals 
are presented in Appendix F. 

T: The college (COLL) norm compares the course to all 
courses in the appropriate college within a 
university (e.g.. Liberal Arts and Sciences). 

U: The OVERALL norm compares the course to all courses 
that have used the CEQ throughout the United States. 
The actual norm decile intervals are presented in 
Appendix G. 

The Interpretation of Results 

The results of both item responses and subscores and total 
scores are reported in deciles (see columns J and K for item 
deciles and columns Q, R, S, T, and U for total and subscore 
deciles). These are the most important results to loqk at to 
adequately understand and interpret the results. Deciles at the 
upper end of the scale (i.e., 7, 8, 9) can be considered highly 
positive and indicative of good instructors and courses, wtiile 
deciles of 0, 1, or 2 can be considered quite negative and indi- 
cative of relatively poor instructors and courses. Deciles 
between 3 and 6 should be considered average. Although no manual 
can set forth an explicit interpretation of results under all 
conditions, for all classes and all instructors, some general 
guidelines can be offered. Individual interpretation can be ex- 
plored in greater detail in conference with the Measurement and 
Research Division staff. Generally, however, the higher an 
instructor's decile, the more satisfied he can be; the lower his 
decile, the more concerned* The research on the CEQ (Spencer & 
Aleamoni, 1969; Swanson & Sisson, 1971; Aleamoni, 1972) and other 
student evaluation systems (Costin, Greenough & Meiiges , 1971) has 
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shown that students are, to a considerable extent, valid judges of 
instructors and instruction; at the least, the opinions of the 
students contribute to a better understanding of the instructional 
process. There may be many factors beyond the instructor's con- 
trol that could affect the student responses. If the instructor 
is aware of such factors operating within the instructional set- 
ting, he should keep them in mind when interpreting the results. 

Student opinion polls may suffer varying degrees of invalidity 
such as: 

1. Each student may have different expectations and 
personal goals for a course. 

2. Students' frames of reference may be 'considerably 
different. 

3. The possibility that student opinions may not be 

a completely valid criterion of teaching effective- 
ness. 

4. A valid concept of good teaching may not be avail- 
able so that the identification of good teaching by 
any means may not be appropriate. 

5. Students may be indifferent or insincere when 
answer,ing the questionnaire. 

However, students do observe teaching more than anyone else. 
Insofar as learning is a function of the attitudes and opinions 
held by learners, student opinions may indicate areas of rapport, 
degrees of communication, or the existence of problems. Informa- 
tion gathered from student questionnaires, therefore, may help 
instructors describe and define the learning environment more con- 
cretely and objectively than they can through other measurements. 
The evidence in research (Spencer & Aleamoni, 1969; Swanson & 
Sisson, 1971) on the subject indicates quite strongly that overall 
they tend to agree with "expert" judges on the effectiveness of 
instruction. 

There are various ways of sampling student opinion. Some 
useful information can be derived simply by determining the number 
of students who agree or disagree with certain statements about 
the course. Or, sometimes, it proves useful to ask students to 
write short essays about the course in order to obtain information 
about their experiences under specific instructional situations. 
Such individualized procedures do not, however, provide an 
opportunity to compare the results of one course with results from 
Another. Measurement is most useful when comparative results are 
available. More adequate interpretation may occur when the data 
have been collected in a standardized fashion, and when many in- 
structors and courses have been measured with the same instrument. 
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Therefore, the primary purpose in the development of this 
questionnaire was to devise an instrument which elicits opinions 
about a standardized set of statements relative to standardized 
aspects of an instructional program and to develop comparative and 
representative norms which would enable an instructor to adequately 
compare his results with the results of other instructors. 

One may also look at the different results for the subscales 
to obtain an indication of major areas of strengths and weaknesses 
in the course. Similarly, one may look at the item results as 
indicative of areas that need improvement. The CEQ is not com- 
pletely diagnostic of teaching or instruction — no instrument can 
be. There are too many specific variables in a learning environ- 
ment which can be scrutinized to measure or evaluate them all on a 
questionnaire, and some would be valuable in one set* ing and not 
in another. The CEQ provides a means whereby some evaluation of 
the teaching process can occur; other means can be arranged, and 
are available such as asking more diagnostic questions in the 
optional item section. This is only one of tne many pieces of 
information which can contribute to a better learning environment. 
It is important to recognize, however, that student opinions are 
in existence and do affect learning— and they do provide a source 
of quite reliable data relative Lo the effectiveness of instruc- 
tion. 

The research on student opinion questionnaires, in general, 
would seem to indicate that there is some reasonable relationship 
between teaching effectiveness and student judgments of this 
effectiveness. However, it is far from perfect; and in some cases, 
the relationship is completely absent. It is, therefore, of 
extreme importance that the results of this questionnaire be con- 
sidered quite tentative. First, one must assume that the question- 
naire is only a tentative indication of "real" student attitudes. 
It is not a perfect measure. Secondly, the questionnaire collects 
some opinions only. It does not sample all opinions that may 
exist about a course. Thirdly, the opinions that develop about a 
course are developed through a variety of causes and not because 
of the instructor alone. 

It is recommended that the results of a one semester sample 
be treated quite tentatively until validated by measures over two 
or more semesters. 

On the basis of the research evidence to date and the CEQ*s 
high reliability, extremely low scores on a particular subscale 
should indicate problem areas needing immediate attention. On th» 
other hand, stable high scores should point to an effective in- 
structional program as viewed by students. 
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If you have any questions regarding your particular results, 
please consult with the Measurement and Research Division, Office 
of Instructional Resources, 307 Engineering Hall, Urbana, Illinois 
61801, (217) 333-3490. 



REFERENCES 



Aleamoni, L. M. A review of recent reliability and validity 

studies on the Illinois course evaluation questionnaire (CEQ) . 
Research Memorandum No. 127. Urbana, Illinois: Measurement 
and Research Division, Office of Instructional Resources, 
University of Illinois, 1972. 

Spencer, R. E. & Aleamoni, L. M. The Illinois course evaluation 
questionnaire: A description of its development and a report 
of some of its results. Research Report No. 292. Urbana, 
Illinois: Measurement and Research Division, Office of 
Instructional Resources, University of Illinois, 1969. 

Swanson, R. A. & Sisson, D. J. The development, evaluation, and 
utilization of a departmental faculty appraisal system. 
Journal of Industrial Teacher Education^ 1971, 9(1), 64-79. 

Costin, F., Greenough, W. T. & Menges , R. J. Student ratings of 
college teaching: Reliability, validity, and usefulness. 
Review of Educational Research^ 1971, 41j 511-535. 



ERIC 



18 



APPENDIX A 

Illinois Coux*se Evaluation Questionnaire 
(Form 66) 
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APPENDIX D 



Decile Norm Cut-off Scores 
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